User Testing in Industry: A Case Study of Laboratory, Workshop, and Field Tests

Applied user testing involves more usability evaluation methods than laboratory tests and is critically dependent upon a number of issues seldom treated in the literature. The development of the system described in this longitudinal, diary-based study evolved around five user tests: a laboratory test, a workshop test, and three field tests. The user tests had a substantial impact on the focus of the entire development effort in that 25% of the primary developer’s time was spent solving problems encountered during the tests. The laboratory test made use of set tasks and was biased toward how tasks were performed with the system, at the expense of what tasks could be performed. The workshop test was more informal and apparently led the users to adopt a more exploratory attitude. Careful arousal and management of the users’ commitment to participate actively proved essential to effective user testing, especially during the field tests.